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(A-69994/RMA) 

SYSTEM AND METHOD FOR ADAPTING CONTENT FOR SENSORY AND PHYSICALLY 
CHALLENGED PERSONS USING EMBEDDED SEMANTIC ELEMENTS IN A PROCEDURALLY 

BASED MESSAGE FILE 

5 

I Claim: 

1. A method for communicating a message to a client device for interaction with a sensory or 
physically challenged recipient, said method comprising steps of: 

(i) identifying an idea to be communicated to said sensory or physically challenged user recipient, 
10 said idea including a message intent which influences the content of the message; 

(ii) collecting and storing a plurality of alternative expressions for said message each said 
alternative expression being associated with a different one of a plurality of possible outputs generated 
by a client device, at least some of said outputs intended to stimulate a different sense of said user; 

(iii) composing a content information set encompassing said message with said message intent 
O 15 from selected ones of said plurality of alternative expressions said message including procedural 

in 

~% components, data components and semantic components identifying the context for which ones or the 

p procedural components and data components will be presented to said recipient, said presentation 

^ including executing ones of said procedural components and rendering of said data components; 

(iv) communicating said content information to said client device for presentation to said 
'~~~4 20 recipient; 

f (v) automatically selecting a particular output to generate from among said plurality of possible 

Ls. outputs; and 

O (vi) executing instructions in said client device to generate said selected output so as to stimulate 

J: a particular one of said user senses. 

• i 

2. The method in claim 1 , wherein said semantic components comprise semantic identifiers. 

3. The method in claim 2, wherein said semantic identifiers comprise semantic flags. 

30 4. The method in claim 2, wherein said semantic components comprise single binary bit identifiers 
used in association with a multi-bit semantic flag mask. 

5. The method in claim 2, wherein said semantic components comprise multi-bit identifiers used 
in association with a multi-bit semantic flag mask. 

35 

6. The method in claim 2, wherein said content information comprises a StoryMail story, and said 
semantic elements comprise semantic flags embedded within the story. 

7. The method in claim 6, wherein said semantic flag elements are selected from the group of 
40 elements consisting of navigation type information elements, and content type information elements. 
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8. The method in claim 6, wherein said method further comprises steps of: 

(a) searching through said story by a procedure executing within a story playback engine within 
the receiving client device to identify procedural components and data components having one or more 
associated semantic flags; and 
5 (b) processing each said content information received according to the existence or 

non-existence of an associated semantic flag, and the type of information identified by the semantic 
flags. 

9. The method in claim 8, wherein said semantic flags identify a navigation type, and a 
10 content type. 

10. The method in claim 1 , further comprising step of: 

soliciting and receiving user input in one or more of a plurality of manners selected from the set 
consisting of: enumerating the available user input sources and selecting from one of the enumerated 
15 input sources, entering choices in words where the manner of input is a combinations of words, 
characters, letters, numbers, sentences, paragraphs, sets of paragraphs, articulated text, so as to 
provide an input for filling out forms. 

1 1 . The method in claim 10, wherein said user senses can be selected from the group of senses 
20 consisting of sight, hearing, touch, smell, taste and combinations thereof. 

12. The method in claim 1, wherein client device possible outputs can include: a display device for 
presenting symbols, text, graphics, and pictures sensible by a user's eyes; an audio output device for 
presenting a sound sensible by a users ears; a tactile output device sensible by a users touch at or 

25 through a skin surface; an electronic signal for coupling to a user skin surface mounted or internally 
implanted sensory transducing device adapted to produce a sensory experience for the user. 

13. The method in claim 1, wherein the step of selecting a particular output to generate from among 
the plurality of possible outputs includes: (i) the selection by the user when the content is received; (ii) 

30 the selection being selected in response to an indicator received with the content; (iii) the selection being 
selected in response to user preferences identified prior to receipt of said content; and (iv) the selection 
being selected in response to client device characteristics. 

14. The method in claim 13, wherein client device characteristics are selected from the group 
35 consisting of: client device hardware characteristics, client device software device characteristics, client 

device firmware characteristics, client device programmatic characteristics, client device data 
characteristics, and combinations thereof. 
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1 5. The method in claim 1 0, wherein when user inputs are solicited, such user inputs are be selected 
from the group of inputs consisting of eye movements, direct sensing of brain signals with electrodes, 
direct sensing of neuromuscular signals, sensing of skin characteristics, and combinations thereof. 

5 16. The method in claim 12, wherein said tactile output device generates a Braille encoded tactilely 
sensible indicia. 




17. The method in claim 1, wherein the plurality of alternative expressions for the idea includes 
symbolic expression. 

10 

18. The method in claim 17, wherein the plurality of alternative expressions for the idea may also 
include a text expression for each content item including a description of all audio and graphical content. 

19. The method in claim 1, wherein the sensory challenged user is selected from the group 
ES 15 consisting of a sight impaired user, a hearing impaired user, a sight and a hearing impaired user. 

g 20. The method in claim 10, wherein the semantic information contained in the message can be 

yj associated with the message and used in conjunction with said solicited user input. 

SJ 20 21 . The method in claim 10, wherein the user input solicitation and enumeration can be performed 
5 by moving a single button to cause the selection to be sequentially highlighted or sequentially articulated 

f. or tactilely identified. 

4= 22. The method in claim 10, wherein the user input solicitation and enumeration are performed by 

}5 25 an act selected from the set of acts consisting of: select from articulated text, selection from items 
enumerated by voice, button pressing, double mouse button clicks, selection based on button press 
during an automated continuous sequential enumeration of the available selectable items, selection 
based on button presses that cause the individual enumeration of selectable items in an order based on 
which buttons are pressed and with an additional button press to perform the actual selection and 
30 combinations thereof. 



23. The method in claim 1, wherein the content adaptation and scaling uses story element 
semantics, and provides a multi-sensory electronic content package for communicating with sensory 
impaired users, the package comprising procedural portions and data portions. 

24. The method in claim 2, wherein there are semantic flags and text behind at least a subset of the 
logical elements of the message to be communicated. 



40 



25. The method in claim 2, wherein said semantic flags allow for automated procedural enumeration 
of the elements needed to communicate the intent of the message and user interaction methods for 
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presentations in a manner conforming to the seiection of a given set of flags of interest and the values 
that said flags of interest must have if each element is to included in the enumeration. 

26. The method in claim 2, wherein the semantic flags' meanings indicate one or more of the 
5 following with respect to identified content: first level complete story message overview, second level 

complete story overview, first level single screen overview, second level single screen overview, contains 
text, contains audio, contains video, contains text backing, contains audio backing, contains video 
backing, is selectable, is visible, selection action description, is played back as audio for this screen, can 
be omitted without losing intent of message, suitable for hearing impaired, suitable for visually impaired, 
10 suitable for people with disabilities of movement, describes what happens when selection is made, 
describes complete list of currently selectable items, is complete text containing the entire intent of 
message. 

27. The method in claim 2, wherein the semantic flags' meanings indicate one or more of the 
q 15 following with respect to identified content: is objectionable for rendering for children under 12 years of 



age, is objectionable for rendering for children under 18 years of age, is objectionable for rendering for 
children under 21 years of age. 



^ 28. The method in claim 2, wherein the semantic flags' meanings indicate one or more of the 

20 following with respect to identified content: contains religion related content, contains Christian related 



content, contains Jewish related content, contains Muslim related content, contains Hindi related content, 
contains Buddhist related content, contains Atheist related content, contains material objectionable to 
men, contains material objectionable to women, and the like. These are merely exemplary and any other 
indicator for particular content type may be applied and coded. 



29. The method in claim 2, wherein semantic flags from additional second group of semantic flags 
are added to a first group of semantic flags to further refine the meaning of the first group of semantic 
flags, said second semantic flags being selected from the set consisting of: as being of a certain priority, 
as being of a certain level, or pertaining to a certain order with respect to the other said semantic flags 

30 which may be set for an element or set of elements. 

30. The method in claim 2, wherein semantic flags are hierarchically structured. 

31 . The method in claim 2, wherein semantic flags are nested. 



25 



35 



32. 



The method in claim 2, wherein semantic flags are hierarchically structured and nested. 



40 



33. The method in claim 10, wherein a given set of semantic flags of interest are isolated and 
identified by the process of performing the equivalent logical operation of a binary logical AND operation 
of the set of binary flags, with a mask value identifying the given set of semantic flags of interest. 
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34. The method in claim 33, wherein the result of the logical AND operation is compared to a set of 
required binary values to determine if the element or elements associated said semantic flags meet the 
criteria for inclusion in the enumeration of selected elements. 

35. The method in claim 33, wherein the semantic flags meet the criteria if the result is found to be 
equal to said required binary values. 

36. The method in claim 33, wherein the semantic flags meet the criteria if the result is found to be 
not equal to the required binary values. 

37. The method in claim 33, wherein the semantic flags meet the criteria if the result is found to 
contain a number of set flag bits having predetermined relation to a reference criteria, said relation being 
selected from the set consisting of: said result being above a given threshold, said result being above 
or equal to a given threshold, said result being below a given threshold, said result being below or equal 
to a given threshold or equal to a given number, said result being of any predetermined logical or 
mathematical relation to said reference criteria. 

38. The method in claim 33, wherein the semantic flags can be further refined as to their respective 
meaning(s), said further identifying including said semantic flag indicating that identified content can be 
used on a particular device, that identified content can be used on a particular operating environment 
or set of operating system environments, that identified content can be used on particular playback 
engine version or versions, and/or that identified content can be used on or in conjunction with a 
particular software application. 



